Tracking a planar patch in three-dimensional space by affine transformation in monocular and binocular vision

-The three-dimensional (3D) motion of a planar surface (or patch) can only be detected from the motion of its projected 2D contour or region image on the plane of vision. Acquirement of corresponding points is the most important procedure. In the past, many methods of how to find corresponding points were published frequently. Affine transformation is used for the matching work. By this transformation, the position, orientation and motion of a planar patch can be calculated. Numerical calculation is very simple in this method because only the mass moments need to be calculated not iterative or exhausting searching. Motion analysis Affine transformation Computer vision I. I N T R O D U C T I O N Detecting the three-dimensional (3D) motion of a planar patch from the two-dimensional (2D) motion of its projected image is one of the most important tasks in computer vision and image processing. One of the prevailing approaches is first to detect the corresponding points and then to analyze their optical flow vectors. ~l-s~ However, it is well-known that the correspondence problem is one of the most difficult problems in image processing; not only because of the discouraging, boring, iterative searching but also because its highly noise-sensitive and ambiguous characteristics make us feel that a new method to detect 3D motion without searching corresponding points should be developed. From reference (4) an idea was obtained about how mass moments were utilized in pattern recognition. Then, a new thought came to mind: "Why not use affine transformation to get motion parameters? ' 's~ So, the method described in reference (4) was modified. In later articles, the way of linking affine transformation with motion parameters will be shown. Before that, details on how the problem of tracking planar patch is formulated, will be described. When the surface normal vector of a planar patch is not parallel to the viewing axis of the viewer (Z-axis), the perceived shape on the image plane is skewed. However, skewing is not the only case that can happen to the projected shape when the patch moves in 3D space. Translation, rotation and scaling should also be considered. After inspecting the projected shape R and R', which correspond to the planar patch before and after motion, that there approximately exists an affine transformation relationship between the two shapes, R and R' is assumed, as shown in Fig. 1. In fact, this relationship is just a kind of point correspondence. Because the affine coefficients are easily derived by the mass moments of the shape R and R' (shown later), iterative searching is no longer required. In addition to easy calculation, mass moments have another excellent characteristic: noise-insensitivity. Finally, some details should be especially noticed here. (1) The reason why the moving rigid body in 3D space was restricted to being planar or approximately planar was so that the occlusion problem did not occur. Only those shapes projected from a planar patch can be utilized to do the matching work by affine transformation. (2) Perspective projection is adopted to model the camera. (3) In this paper, the use of one camera (monocular vision) or two cameras (binocular vision) to track a planar patch in 3D space is attempted. (4) For simplicity, the projected image should be thresholded first to get a bilevel image. Level white for the projected shape and black for the background. (5) Assume that it is known that the projected shapes come from the same planar patch in 3D space (i.e. recognition has been done previously). 2. U S I N G MASS M O M E N T S T O C A L C U L A T E A F F I N E T R A N S F O R M A T I O N C O E F F I C I E N T S As shown in Fig. 1, there are two shapes R and R' on the image plane. Both of them can be considered as different projected shapes of the same object planar patch at different positions and orientations. Now, a pixel point p = (x, y) in region R can map to another pixel point p' = (x', y') in region R' via the